T owards Latency-aware 3D Streaming Perception for Autonomous Driving Jiaqi Peng 1,2, Tai Wang 2, Jiangmiao Pang 2 and Y uan Shen 1,2 Abstract -- Although existing 3D perception algorithms have demonstrated significant improvements in performance, their deployment on edge devices continues to encounter critical challenges due to substantial runtime latency. We propose a new benchmark tailored for online evaluation by considering runtime latency. Based on the benchmark, we build a Latency-A ware 3D Streaming Perception (LASP) framework that addresses the latency issue through two primary components: 1) latency-aware history integration, which extends query propagation into a continuous process, ensuring the integration of historical feature regardless of varying latency; 2) latency-aware predictive detection, a module that compensates the detection results with the predicted trajectory and the posterior accessed latency. By incorporating the latency-aware mechanism, our method shows generalization across various latency levels, achieving an online performance that closely aligns with 80% of its offline evaluation on the Jetson AGX Orin without any acceleration techniques. I. INTRODUCTION 3D perception is an essential capability for autonomous vehicles and provides the foundation for subsequent prediction and planning [1], [2]. The past few years have witnessed the rapid advancement of 3D perception algorithms [3]- [5]. In particular, one of the most popular settings, i.e. [6], [7], only using multiple cameras, can achieve performance that is comparable with LiDAR-based methods [8], [9] with effective BEV -based paradigms [5], [6].

Last month, NVIDIA unleashed the next-generation edge computing hardware device branded as Jetson AGX Orin at GTC. Courtesy of Nvidia, I was fortunate enough to get a Jetson AGX Orin Developer Kit to evaluate and experiment with it. The Jetson AGX Orin Developer Kit has everything you need to run AI inference at the edge with ultra-low latency and high throughput. As a successor to the most powerful Jetson AGX Xavier, AGX Orin packs a punch. The developer kit comes with a carrier board that makes it easy to connect various peripherals. The Jetson AGX Orin Developer Kit comes with a preview of JetPack SDK 5.0, which is based on the Ubuntu 20.04 root filesystem and Linux Kernel 5.10.

Learn more about what comes next. At its GTC event today, Nvidia debuted Jetson AGX Orin, which it claims is among the world's smallest, most energy-efficient AI accelerators for robotics and edge devices. Built on Nvidia's Ampere architecture, Jetson AGX Orin delivers a claimed 6 times the processing power of its predecessor, Jetson AGX Xavier: up to 200 trillion operations per second. The launch of Jetson AGX Orin comes as the market for edge computing hardware climbs steeply during the pandemic. Edge computing brings computation, data storage, and power closer to the source of data generation, allowing for more immediate insights from connected systems.